Time Scales in Evolutionary Dynamics 
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Evolutionary game theory has traditionally assumed that all individuals in a population interact with each 
other between reproduction events. We show that eliminating this restriction by explicitly considering the time 
scales of interaction and selection leads to dramatic changes in the outcome of evolution. Examples include 
the selection of the inefficient strategy in the Harmony and Stag-Hunt games, and the disappearance of the 
coexistence state in the Snowdrift game. Our results hold for any population size and in more general situations 
with additional factors influencing fitness. 



Evolutionary game theory is the mathematical framework 
for modelling evolution in biological, social and economical 
systems fl','2; '3*1, and is deeply connected to dynamical sys- 
tems theory and statistical mechanics |4, 5, 6, 7, 8, 9, 10, 11]. 
In the standard setup of evolutionary game theory, strategies 
available for the game are represented by a fraction of indi- 
viduals in the population. Individuals then interact accord- 
ing to the rules of the game, and the so earned payoffs deter- 
mine the frequencies of the next generation (i.e., payoffs rep- 
resent reproductive fitness). Customarily, most evolutionary 
game studies make the additional assumption that individuals 
play many times and with all other players before reproduc- 
tion takes place, so that payoffs, equivalently fitness, are given 
by the mean distribution of types in the population. This is 
also the situation for the so called round-robin tournament, 
in which each individual plays once with every other. Both 
hypotheses, common in biological evolution, implies that se- 
lection occurs much more slowly than the interaction between 
individuals. Although recent experimental studies show that 
this may not always be the case in biology 0111 El, It IS 
clear that in cultural evolution or social learning the time scale 
of selection is much closer to the time scale of interaction. The 
effects of this mixing of scales cannot be disregarded fl3l . and 
then it is natural to ask about the consequences of the above 
assumption and the effect of relaxing it. Though the main field 
of application of our work is social and cultural evolution, we 
maintain the usual language of evolutionary biology, to avoid 
introducing new terminology. 

In this Letter, we show that rapid selection affects evolu- 
tionary dynamics in such a dramatic way that for some games 
it even changes the stability of equilibria. In order to make 
explicit the relation between selection and interaction time 
scales, we use discrete-time dynamics. We follow Moran dy- 
namics 1 16], as this is the proper way to describe evolution of 
discrete generations in the field of population dynamics 
Specifically, we choose the frequency-dependent version of 
the Moran dynamics introduced by ] 18], which allows to con- 
sider an evolutionary game in this dynamical context: N indi- 
viduals interact by playing a game and reproduce by selecting 
one individual, with probability proportional to the payoff, to 
duplicate and substitute a randomly chosen individual. The 
payoff of every player is set to zero after each reproduction 



event, and this two-step cycle is repeated until the population 
eventually stabilizes. This stochastic dynamics is discrete in 
both population and time, while keeping the population size 
constant over time. Interestingly, this microscopic dynamics 
leads to a_difference equation that has been proposed as an 
adjusted ]!] or discrete-time f?] analogous of the replicator 
equation, widely used in evolutionary game theory (see ]3] 
for a recent, detailed discussion of this issue). Additionally, 
we note that for social applications, reproduction may be also 
interpreted as a learning process, in which individuals do not 
die but instead change the way they behave or their strategies. 

Time scales enter the dynamics through the interaction step, 
affecting the way fitness is obtained. We introduce a new in- 
teraction scheme, by allowing an integer number s of ran- 
domly chosen pairs of individuals to play consecutively the 
game, between reproduction events. Thus, s equals the ratio 
between selection and interaction time scales. This is the cru- 
cial parameter in our model. The limit value of s = 1 means 
that both time scales are equal; greater finite values, s > 1, 
correspond to the selection time scale being slower than the 
interaction time scale, and the limit value of s — > oo recovers 
the round-robin procedure. In fact, the equivalence of the limit 
s — > oo to the round-robin scheme points to the latter being 
a form of 'mean-field' theory, in which individuals reproduce 
so slowly that it makes sense to replace pairwise interactions 
by the interaction with the ' average player' . 

As for the games, we will consider the important case of 
symmetric 2x2 games, in which the payoffs are given by the 
following matrix 
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whose rows give the payoff obtained by each strategy when 
confronted with the other or itself, and a, b,c,d > 0. Let n 
be the number of individuals using strategy 1, also referred 
as type 1 individuals. After each reproduction event n may 
stay the same, increase by one, or decrease by one. Consid- 
ering the definition of the dynamics, the corresponding tran- 
sition probabilities will depend on the fitness earned by each 
type during the interaction step and on their frequencies. As 
both quantities will depend ultimately on n, we have a Markov 
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process with a tridiagonal transition matrix (i.e., a birth-death 
process L19.I ) whose non-zero coefficients are 
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and P„_„ = 1 — Pn,n-i — Pn.n+i- Fi is the payoff obtained 
by all players of type i, and E{-\n) denotes the expected value 
conditioned to a population of n individuals of type 1 . 

We stress that the parameter s enters through the expected 
values of the relative fitness of each type (|2}- Indeed, if we 
restrict ourselves to the limit s ^ 00, these expected values 
are given directly by the pairing probabilities and the payoffs 
corresponding to each pair 
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n{n — l)a + n{N — n)b 



n{n - l)a + n{N - n){b + c) + (iV - n){N - n - l)d 
as would be obtained by the round-robin scheme. However, 
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as we will see below, finite values of s often lead to results 
completely different from this special case. 

The solution to the birth-death process we have just de- 
scribed can be obtained in a standard manner CI. Denoting 
by pn the fixation probability of type 1 (i.e. the probability of 
ending up in a population with all individuals of type 1 ) when 
starting from a population with n players of this type, we have 



Pn = Pn,n-lPn-l + P-n .7iPn + Pn (4) 

with po = and = 1. The solution to this equation is 
given by 
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with Qi = 1. As stated above, the interesting case arises for 
finite values of the parameter s. For general s, a straightfor- 
ward combinatorial analysis of all the possible sequences of s 
pairings leads to 



E 



EE 

i=0 j=Q 



- sln^-^n - iy{N - ny-'{N 



2ai + b{s — i — j) 



i\j\{s-i-jy.{N{N -i)y 



2ai + 2dj + (& + c)(s 



(6) 



r 



This lengthy combinatorial expression reduces, in the limit 
case s — 1 of extremely rapid selection, to 
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The above equations are the first hint of the effect of time 
scales. Indeed, by noting that, for this extreme case, only the 
coefficients of the skew diagonal of Q appear in we reach 
the surprising conclusion that if the time scale of selection 
equals that of interaction, the evolutionary outcome of any 
game will be determined solely by the performance of each 
strategy when confronted with the other, and independently 
of the results when dealing with itself. However, as we will 
now see, there are another non-trivial, important differences. 

To make our study as general as possible, we have analyzed 
all twelve non-equivalent symmetric 2x2 games |20|. These 
games can be further classified into three categories, accord- 
ing to their Nash equilibria and their dynamical behavior un- 
der the replicator dynamics with round-robin interaction: 

I. There are six games with a > c and 6 > d, or a < c and 
b < d. They have a unique Nash equilibrium, corresponding 



to the dominant pure strategy. This equilibrium is the global 
attractor of the round-robin replicator dynamics. 

II. There are three games with a > c and b < d. They have 
several Nash equilibria, one of them with a mixed strategy. 
With the round-robin replicator dynamics, this mixed strategy 
equilibrium is an unstable point, which acts as the boundary 
between the basins of attraction of the two pure strategies, 
which are the attractors. 

III. The remaining three games have a < c and b > c. They 
have several Nash equilibria, one of them with a mixed strat- 
egy. This mixed strategy equilibrium is the global attractor of 
the round-robin replicator dynamics. The two pure strategies 
are unstable in this case. 

Let us first consider an example of class I, namely the Har- 
mony game |^ (a = 1, 6 = 0.25, c = 0.75, d = 0.01). This 
is a no-conflict game, in which all players obtain the maxi- 
mum payoff by following strategy 1. As Fig.fHa) shows, this 
is the result for large values of s, with a fixation probability 
p„ « 1 for almost all n. On the other hand. Fig. [fla) also 
shows that, for small s, strategy 2, i.e., the inefficient (in the 
sense of lowest payoff) one, is selected by the dynamics, un- 
less starting from initial conditions with almost all individuals 
of type 1 . 

For class II, a good paradigm is the Stag-Hunt game i23l 
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FIG. 1: Fixation probabilities in the games (a) Harmony (a = 1, 
b = 0.25, c = 0.75, d = 0.01) and (b) Stag-Hunt (a = 1, 6 = 0.01, 
c = 0.8, d = 0.2), for s equal to 1 (o), 5 (□), 10 (A), 100 (+), or 
cxj (x). Note that, in figure (a), curves overlap for s = 10, 100 
and oo. Population size TV = 100. 

(a — 1, b — 0.01, c = 0.8, d — 0.2), which is a coordina- 
tion game: Strategy 1 maximizes the mutual benefit, whereas 
strategy 2 minimizes the risk of loss, and the conflict results 
from having to choose between these two options. As Fig. 
[Qb) reveals, the round-robin result is obtained for large s: 
both strategies are attractors, with the basin boundary located 
at the frequency corresponding to the mixed strategy equilib- 
rium, i.e. X — {d — b)/{a — c + d — b) « 0.49. However, for 
small values of s this boundary shifts to greater frequency val- 
ues, thus reflecting an advantage of strategy 2. In the extreme 
s ~ 1 case this strategy becomes the unique attractor. 

It is interesting to note that Fig.^shows that there is not a 
general crossover ats~N. In the Harmony game, the round- 
robin regime is mostly reached for s ~ 10 <C iV, whereas in 
the Stag-Hunt game this does not happen until s ~ 100 — N. 

Finally, let us consider the Snowdrift game i23ll (a — 1, 
6 = 0.2, c — 1.8, d — 0.01) as an example of class III. This 
is also a dilemma game, as each player has to choose between 
strategy 1, which maximizes the population gain, and strategy 
2, which gives individuals the maximum payoff by exploit- 
ing the opponent. With round-robin dynamics both strate- 
gies coexists in the long run, with frequencies corresponding 
to the mixed strategy equilibrium. However, our dynamics 
can never maintain coexistence indefinitely, because by con- 
struction one of the absorbing states (all players of type 1 
or all of type 2) will be reached sooner or later with proba- 
bility 1. Nonetheless, it is possible to study the duration of 
metastable states by using the mean time in each population 
state before absorbtion, i„ d- Figure El shows the results 
for two values of s and a broad range of initial conditions. 
For s large (s — 100), the population stays for a long time 
near the value corresponding to the mixed strategy equilib- 
rium X = {d — b) /{a ~ c + d — b) « 0.19, independently 
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FIG. 2: Mean time before fixation in the Snowdrift game (a = 1, 
b = 0.2, c = 1.8, d = 0.01), for s equal to 5 (a) and 100 (b). Initial 
values of n equal to 20 (o), 50 (A) and 80 (+). Note that curves in 
(b) overlap. Population size A*' = 100. 

of the initial number of type 1 individuals. A smaller value 
of s = 10 (not shown) induces a shift of the metastable equi- 
librium to smaller values of n, again almost independently of 
the initial conditions. Finally, for an even smaller value of s 
{s = 5), there is no metastable equilibrium, but a fluctuation 
towards the x = absorbing state, which clearly depends on 
the initial conditions. 

Having given examples of all three classes, we will summa- 
rize the rest of our study by saying that the remaining 2x2 
games behave in a similar way, with rapid selection (small s) 
favoring in all cases the type that has the greatest coefficient in 
the skew diagonal of the payoff matrix. For the remaining five 
games of class I this results in a reinforcement of the domi- 
nant strategy (the Prisoner's Dilemma being a prominent 
example). The other two games of class II exhibit once again 
a displacement of the basins of attraction, whereas the other 
two class III games display the suppression of the coexistence 
state in favor of one of the strategies. We thus see that rapid 
selection leads very generally to outcomes entirely different 
from those of round-robin dynamics. 

It is important to realize that our results do not change qual- 
itatively with the system size. Considering for instance the 
Stag-Hunt game, the change in the basins of attraction is prac- 
tically independent of the population size. The main effect of 
working with larger sizes is a steeper transition between the 
basins of attraction. Indeed, due to the inherent stochastic- 
ity of finite population sizes, smaller populations have a more 
blurred basin boundary, with points in each basin having an in- 
creasing non-zero probability of reaching the other basin 1 251 . 
Our results for all other symmetric 2x2 games are equally ro- 
bust. In fact, for very rapid selection, s ~ 1, the limit N ~* oo 
of the transition probabilities, Eq. (0, shows that they depend 
only on the frequencies of both types. 

It could be argued that in our model only s pairs of indi- 
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FIG. 3: Fixation probability in the Stag-Hunt game (a = 1, b — 
0.01, c — 0.8, d — 0.2) with a background of fitness /(, equal to 0.1 
(a) and 1 (b). Values of s: 5 (□), 10 (A) 100 (+). Population size 
N = 100. 



viduals play in each round, resulting in a very small effec- 
tive population, this being the fundamental cause of the re- 
ported results. To probe into this issue, we have introduced 
a background of fitness [J^ .18il . so that every player has an 
intrinsic probability of being selected, regardless of the out- 
come of the game, and thus guaranteeing a population of N 
players. Indeed, in most applications, agents interact through 
more than one type of game and there are external contribu- 
tions to fitness(environmental factors, fashions or media influ- 
ence in a social context, etc.). Let /b be the normalized fitness 
background, so that each individual has a background of fit- 
ness s/f, /N before selection takes place; fb — I means that 
the overall fitness coming from the game and from the back- 
ground are approximately equal, for every value of s and N. 
Figure|3]shows the results for the Stag-Hunt game. A small fit- 
ness background of /b = 0.1 gives fixation probabilities very 
similar to those with fb — (Fig. Qb)). For larger values, 

= 1, the displacement of the basin boundary is smaller, but 
still perfectly noticeable. And a very large fitness background, 
fb }t 10 (not shown), drives the dynamics to random selection 
for every value of s, because in this case the influence of the 
game is almost negligible. Again, for the remaining symmet- 
ric 2 x 2 games, our conclusions remain valid as well in the 
presence of a background of fitness. Consequently, our results 
are not merely due to a finite size effect of a small effective 
population of players. 

In summary, we have proven that considering indepen- 
dent interaction and selection time scales leads to highly non- 
trivial, counter-intuitive results. We have demonstrated the 
generality of this conclusion by considering all symmetric 
2x2 games and showing that rapid selection may lead to 
changes of the asymptotically selected equilibria, to changes 
of the basins of attraction of equilibria, or to suppression of 
long-lived metastable equilibria. This result has major im- 



plications for applying evolutionary game theory to model a 
specific problem, as the assumption of slow selection and con- 
sequently of round-robin dynamics may or may not be correct. 
Indeed, as the example in IllSr shows, rapid selection may lead 
to the understanding of problems where Darwinian, individual 
evolution was thought not to play a role because round-robin 
dynamics was used. We envisage that successful modelling in 
rapidly changing environments, such as social or (sub-)culture 
dynamics, will need a careful consideration of the involved 
time scales along the lines discussed here. 
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